Image Scaling

ABSTRACT

The present invention relates to an apparatus, method for adjusting depth characteristics of a three-dimensional image for correcting for errors in perceived depth when scaling the three-dimensional image, the method comprising: receiving three-dimensional image information comprising a stereoscopic image including a first image and a second image, the stereoscopic image having depth characteristics associated with an offset of the first and second images; determining a scaling factor indicative of a scaling for converting the stereoscopic image from an original target size to a new size; determining at least one shifting factor for varying the depth characteristics, the at least one shifting factor indicative of a relative shift to be applied between the first and the second images, wherein the at least one shifting factor is determined in accordance with the scaling factor and at least one depth parameter derived from the depth characteristics; and performing the relative shift between the first and second images in accordance with the shifting factor for adjusting the offset of the first and second images.

FIELD OF THE INVENTION

The present invention relates to a method, apparatus, and computerprogram product for use in image scaling. In particular, the presentinvention relates to depth perceptive correction of scaled stereoscopicthree-dimensional images on different sized displays.

BACKGROUND OF THE INVENTION

Display technologies are integral to most electronic devices, being usedboth for watching media such as television or films and for graphicaluser interfaces for computers, mobile phones and other electronicdevices.

In recent years, the use of three-dimensional technology in suchdisplays has been gathering momentum. In particular, the use of suchthree-dimensional technologies for television broadcasting has become ofparticular interest. The basic technology behind three-dimensionalimaging is well known and well established, dating back to the early1900s. Three-dimensional moving images have also been around for manyyears, but have not yet been utilised much in consumer electronicdevices.

Three-dimensional imaging works by tricking the eye into perceivingdepth information through two or more images. There are variousestablished techniques for achieving this, the most popular of whichutilise two images and are called stereoscopic techniques. Stereoscopictechniques utilised for moving images such as television involvedisplaying each image to be viewed by the viewer as two images; oneimage arranged to be viewed by the right eye and one image arranged tobe viewed by the left eye.

The left and right images differ slightly such that when they reach eacheye the viewer can extract depth information from the images. Such depthinformation is created using two (slightly) different images. Eachportion of the left and right images contains views of objects that arecaptured from subtly different perspectives. As a result, the offset ofthe views differ by a certain number of pixels, in accordance with thedepth to be perceived by the viewer.

In order to allow for such a three-dimensional imaging technique towork, a mechanism is required to separate the images for the left eyefrom images for the right eye. There are several such techniques. Onetechnique is to display the images alternately in quick succession anduse a mechanism synchronized to the display to control which eye seesthe display only when the appropriate image is displayed. Othertechniques include the use of passive polarization glasses and apolarized display, which allow each eye to see part of the screen forall of the time.

When broadcasting three-dimensional images, for example as part oftelevision broadcast, the perception of depth that is associated withthree-dimensional images is achieved by a pair of similartwo-dimensional images captured from slightly different perspectives andthus slightly offset from each other. The offset of the two images,which in turn determines the perceived depth of the image, is determinedin accordance with a fixed image size before transmission. For example,the depth may be determined for displaying on a 48-inch television. Ifthe received television signal is then displayed on a 32-inchtelevision, the image information will be scaled by ⅔rds for beingdisplayed on the 32-inch television.

SUMMARY OF THE INVENTION

It has been noted by the inventors of the present invention that theperception of depth does not scale proportionally to the scaling of thescreen size. The closer an object is perceived as being to a screen, thesmaller the variation in perceived depth that occurs when scaling animage. By comparison, the further an object is perceived as being awayfrom the screen the larger the perceived depth will be altered simply bychanging the size of the images. Referring to the previous example,where the televisions' sizes changes from 48 inch to 32 inch, andtherefore scaled by ⅔rds, the perceived depths are not scaledconsistently or scaled by ⅔rds (except for one unique depth).Consequently, the depth information is distorted when scalingthree-dimensional image pairs.

One solution is to transmit multiple signals corresponding to differentimage sizes.

There is provided in accordance with an embodiment of the presentinvention a method for adjusting depth characteristics of athree-dimensional image for correcting for errors in perceived depthwhen scaling the three-dimensional image, the method comprising:receiving three-dimensional image information comprising a stereoscopicimage including a first image and a second image, the stereoscopic imagehaving depth characteristics associated with an offset of the first andsecond images; determining a scaling factor indicative of a scaling forconverting the stereoscopic image from an original target size to a newsize; determining at least one shifting factor for varying the depthcharacteristics, the at least one shifting factor indicative of arelative shift to be applied between the first and the second images,wherein the at least one shifting factor is determined in accordancewith the scaling factor and at least one depth parameter derived fromthe depth characteristics; and performing the relative shift between thefirst and second images in accordance with the shifting factor foradjusting the offset of the first and second images.

Further, in accordance with an embodiment of the present invention, theat least one depth parameter used to determine the at least one shiftingfactor relates to a variation in the depth of the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the variation corresponds to an average variation in thedepth of the stereoscopic image.

Additionally, in accordance with an embodiment of the present invention,the average variation is determined in accordance with the followingequation:

$( \frac{{{maximum}\mspace{14mu} {disparity}} + {{minimum}\mspace{14mu} {disparity}}}{2} ) \times ( {1 - {{scaling}\mspace{14mu} {factor}}} )$

wherein the disparity is a measure of an offset between a pixel in thefirst and second images of the stereoscopic image.

Moreover, in accordance with an embodiment of the present invention, theaverage variation is determined in the depth domain and transformed tothe disparity domain for determining the shifting factor.

Further, in accordance with an embodiment of the present invention, theat least one shifting factor is determined by an iterative process forminimising errors in the offset of the first and second images of thestereoscopic image when scaling the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the iterative process utilises information related to theusage of different depth values.

Additionally, in accordance with an embodiment of the present invention,the at least one depth parameter used to determine the at least oneshifting factor includes a parameter indicating a most common depth ofthe depth characteristics of the stereoscopic image.

Moreover, in accordance with an embodiment of the present invention, thethree-dimensional image information includes the at least one depthparameter.

Further, in accordance with an embodiment of the present invention, thestereoscopic image comprises a plurality of regions and the step ofdetermining the at least one shifting factor further comprisesdetermining a shifting factor for one or more of the regions, and thestep of performing a relative shift further comprises performing arelative shift for each of the one or more regions in accordance withthe shifting factor determined for each respective one or more region.

Still further, in accordance with an embodiment of the presentinvention, the step of determining the shifting factor further comprisessplitting the stereoscopic image into regions.

Additionally, in accordance with an embodiment of the present invention,the three-dimensional image information includes information definingthe regions of the stereoscopic image.

Moreover, in accordance with an embodiment of the present invention, themethod further comprises performing a smoothing operation on boundariesbetween regions.

Further, in accordance with an embodiment of the present invention, thesmoothing operation involves low-pass filtering.

Still further, in accordance with an embodiment of the presentinvention, the method further comprises receiving furtherthree-dimensional image information comprising a further stereoscopicimage including a first image and a second image, the furtherstereoscopic image having depth characteristics associated with anoffset of the first and second images of the further stereoscopic image,wherein the stereoscopic image and the further stereoscopic imagecorrespond to frames of a stream of image information and the furtherstereoscopic image corresponds to a frame of the stream preceding thefurther stereoscopic image; and performing a smoothing operation on thedepth characteristics in order to smooth the transition of depthcharacteristics between frames.

Additionally, in accordance with an embodiment of the present invention,the smoothing operation comprises applying a low pass filter to thedepth information.

Moreover, in accordance with an embodiment of the present invention, thedepth parameters are determined in accordance with a depth map includedin the three-dimensional image information, the depth map indicating arelative offset between the first and the second images at a pluralityof points across the stereoscopic image.

Further, in accordance with an embodiment of the present invention, themethod further comprises generating a depth map from the stereoscopicimage, wherein the depth parameters are determined in accordance withthe depth map, the depth map indicating a relative offset between thefirst and the second images at a plurality of points across thestereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the three-dimensional image information is received over anetwork from a headend.

Additionally, in accordance with an embodiment of the present invention,the first and second images are each shifted relative to one another byhalf of the shifting factor for correcting depth perception due toscaling of the three-dimensional image.

In accordance with a further embodiment of the invention there isprovided an apparatus for adjusting depth characteristics of athree-dimensional image for correcting for errors in perceived depthwhen scaling the three-dimensional image, the apparatus comprising: aninput port arranged to receive three-dimensional image informationcomprising a stereoscopic image including a first image and a secondimage, the stereoscopic image having depth characteristics associatedwith an offset of the first and second images; and a processor arrangedto: determine a scaling factor indicative of a scaling for convertingthe stereoscopic image from an original target size to a new size;determine at least one shifting factor for varying the depthcharacteristics, the at least one shifting factor indicative of arelative shift to be applied between the first and the second images,wherein the at least one shifting factor is determined in accordancewith the scaling factor and at least one depth parameter derived fromthe depth characteristics; and perform the relative shift between thefirst and second images in accordance with the shifting factor foradjusting the offset of the first and second images.

Further, in accordance with an embodiment of the present invention, theat least one depth parameter used to determine the at least one shiftingfactor relates to a variation in the depth of the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the variation corresponds to an average variation in thedepth of the stereoscopic image.

Additionally, in accordance with an embodiment of the present invention,the processor determines the average variation in accordance with thefollowing equation:

$( \frac{{{maximum}\mspace{14mu} {disparity}} + {{minimum}\mspace{14mu} {disparity}}}{2} ) \times ( {1 - {{scaling}\mspace{14mu} {factor}}} )$

wherein the disparity is a measure of an offset between a pixel in thefirst and second images of the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the processor determines the average variation in the depthdomain and transforms the average variation to the disparity domain fordetermining the shifting factor.

Additionally, in accordance with an embodiment of the present invention,the processor determines the at least one shifting factor by aniterative process for minimising errors in the offset of the first andsecond images of the stereoscopic image when scaling the stereoscopicimage.

Moreover, in accordance with an embodiment of the present invention, theiterative process utilises information related to the usage of differentdepth values.

Further, in accordance with an embodiment of the present invention, theat least one depth parameter used to determine the at least one shiftingfactor includes a parameter indicating a most common depth of the depthcharacteristics of the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the three-dimensional image information includes the at leastone depth parameter.

Additionally, in accordance with an embodiment of the present invention,the stereoscopic image comprises a plurality of regions and theprocessor determines a shifting factor for one or more of the regions,and performs the relative shift for each of the one or more regions inaccordance with the shifting factor determined for each respective oneor more region.

Moreover, in accordance with an embodiment of the present invention, theprocessor determines the shifting factor by splitting the stereoscopicimage into regions.

Further, in accordance with an embodiment of the present invention, thethree-dimensional image information includes information defining theregions of the stereoscopic image.

Still further, in accordance with an embodiment of the presentinvention, the processor is further arranged to perform a smoothingoperation on boundaries between regions.

Additionally, in accordance with an embodiment of the present invention,the smoothing operation involves low-pass filtering.

Moreover, in accordance with an embodiment of the present invention, theapparatus is further arranged to: receive further three-dimensionalimage information comprising a further stereoscopic image including afirst image and a second image, the further stereoscopic image havingdepth characteristics associated with an offset of the first and secondimages of the further stereoscopic image, wherein the stereoscopic imageand the further stereoscopic image correspond to frames of a stream ofimage information and the further stereoscopic image corresponds to aframe of the stream preceding the further stereoscopic image; andperform a smoothing operation on the depth characteristics in order tosmooth the transition of depth characteristics between frames.

Further, in accordance with an embodiment of the present invention, thesmoothing operation comprises applying a low pass filter to the depthinformation.

Still further, in accordance with an embodiment of the presentinvention, the processor determines the depth parameters in accordancewith a depth map included in the three-dimensional image information,the depth map indicating a relative offset between the first and thesecond images at a plurality of points across the stereoscopic image.

Additionally, in accordance with an embodiment of the present invention,the processor is further arranged to generate a depth map from thestereoscopic image, wherein the depth parameters are determined inaccordance with the depth map, the depth map indicating a relativeoffset between the first and the second images at a plurality of pointsacross the stereoscopic image.

Moreover, in accordance with an embodiment of the present invention, thethree-dimensional image information is received over a network from aheadend.

Further, in accordance with an embodiment of the present invention, theprocessor is arranged to shift the first and second images relative toone another by half of the shifting factor for correcting depthperception due to scaling of the three-dimensional image.

In accordance with yet another embodiment of the invention there isprovided a carrier medium carrying computer readable code forcontrolling a suitable computer to carry out the method as describedabove.

In accordance with a further embodiment of the invention there isprovided a carrier medium carrying computer readable code forconfiguring a suitable computer as the apparatus as described above.

Embodiments of the invention provide an easy and cost effective meansfor adjusting the depth characteristics of an image when varying theimage size. In particular, embodiments of the invention provide a meansfor adjusting the depth characteristics with a minimum amount ofprocessing.

Embodiments of the invention globally shift the stereoscopic images tocompensate for errors in depth characteristics. Furthermore, the globalshift aims to take into consideration characteristics of the image. Suchcharacteristics could include the maximum and minimum depth values.Alternatively, the average or most common depth value could be utilisedin order to minimise the errors in depth seen across the image.Furthermore, it would also be possible to attempt to minimise errors inthose depths that will be most noticeable to the viewer, such as thoseperceived as being close to the viewer.

Embodiments of the present invention provide improved depth shiftingcompensation techniques. Such improvements involve splitting the imageinto portions or sections, and applying appropriate techniques toportions/sections of the image. For example, more complex depthcompensation could be applied to those portions that are of visualsignificance to the viewer. This would therefore allow for savings interms of processing power to be achieved for portions of littlesignificance, as a simple cost-efficient processing means could beapplied to those portions. Further techniques leading to furtherimprovements in such depth perspective correction include smoothing ofimages for errors both between portions and between image frames.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is an illustration of a system constructed and operative inaccordance with an embodiment of the present invention;

FIG. 2 shows a ray diagram illustrating how depth is perceived in athree-dimensional stereoscopic image; and

FIG. 3 is an illustration of a process performed by the system of FIG. 1in accordance with an embodiment of the present invention.

In the description and drawings like reference numerals refer to likeparts.

DETAILED DESCRIPTION OF EMBODIMENTS

Reference is now made to FIG. 1, which provides an overview of thesystem of a first embodiment of the present invention.

In FIG. 1, a television broadcaster 101 provides a three-dimensionalimage, represented as a pair of two-dimensional images, to betransmitted to a transmitter 102, which then transmits thethree-dimensional image as a wireless signal to a receiver 103. Thetelevision broadcaster 101 and transmitter 102 shall hereinafter bereferred to as the head-end 101, 102 of the system.

The receiver 103 is connected to a set-top box 104. The set-top box 104then performs any required processing on the received signal. Theprocessed signal is then provided to a display 105 for being displayed.The processing carried out by the set-top box will include means forcompensating for the errors that occur when scaling the images of thereceived signal to the screen size of the display 105. The set-top box104 is arranged to perform pre-display depth processing. Suchpre-processing of the depth characteristics helps to minimise thedistortion that occurs when scaling images, this is discussed in moredetail below.

In order to fully appreciate how the set-top box performs suchprocessing it is firstly necessary to fully appreciate the signal thatis being processed. An overview of the stereoscopic image signal to beprocessed is therefore set out below.

In this first embodiment of the present invention, a stereoscopicthree-dimensional image stream is transmitted in the form of aconsecutive set of stereoscopic images, transmitted in the order to beviewed. Each stereoscopic image of the image stream comprises twoimages, one image arranged to be viewed by the left eye, and one imagearranged to be viewed by the right eye. In this embodiment, the image tobe viewed by the left eye is placed in the image stream before the imageto be viewed by the right eye. However, it will be appreciated that thisordering could be reversed.

In alternative embodiments of the invention polarisation techniques orparallax barrier techniques may be used. For example, in the case ofpolarisation techniques both images of a stereoscopic image can beviewed simultaneously, and separation of the two images is achievedusing polarised glasses rather than shutter glasses.

In accordance with embodiments of the invention, the perception of depthis achieved from two images that are both subtly different from eachother and offset relative to one another. There is a specific pixeloffset for pixels of the stereoscopic image. Pixels of the image to beobserved are offset between the left and right images in terms of anumber of pixels. A larger pixel offset results in a greater deptheither behind or in front of the screen (depending on the direction ofthe offset).

The human visual system (HVS) interprets the location in depth of anobject based on the relative offsets of the two images, with an absenceof offset (i.e. a zero-offset) giving a depth location at the depthposition of the screen. Image pixels typically represent part of one (orpotentially more) objects, and the HVS matches the two object views fromeach eye. Among a number of cues used to determine depth is thebinocular disparity that is the offset between the objects. Thisdisparity calculation can be modelled as a ray diagram as shown in FIG.2. FIG. 2 shows two intersecting lines L_(l), L_(r) one line L_(l)passing through the left eye E_(l) and a pixel P_(l) intended to beviewed by the left eye E_(l) and the other line passing through theright eye E_(r) and the equivalent pixel P_(r) intended to be viewed bythe right eye E_(r). The point of intersection P_(d) then represents thedepth at which the object is perceived.

The system of the first embodiment of the present invention aims tolargely correct for distortions in the depth characteristics of astereoscopic image when scaling a stereoscopic image in a simple manner,which minimises the processing required by the set-top box. Inparticular, this embodiment of the present invention achieves this byperforming a linear global shift on pixels of the stereoscopic imagepair. That is, pixels of the left and right images of the stereoscopicimage are shifted with respect to one another by a fixed amount, whichcould be a non-integral value. Thus, when the stereoscopic image isdisplayed, the depth of the stereoscopic image is perceived as beingrelatively accurate.

When the presently described embodiment of the invention performs thisshifting it takes into consideration the range of depths of thestereoscopic image, and shifts the stereoscopic image pairs in order tominimize the errors across the range. This is done by calculating aglobal offset that will position the average offset at the same value ason the original screen size. This offset is calculated by subtractingthe average of the scaled offsets from the average of the originaloffsets. This can be summarized by the simplified equation 1:

$\begin{matrix}{{{{Pixel}\mspace{14mu} {offset}} = ( {{{Average}\mspace{14mu} {{Disparity}({original})}} - {{Average}({new})}} )}{{where}\text{:}}{{{Average}\mspace{14mu} {{Disparity}({Original})}} = \frac{( {{\min \mspace{11mu} {{disparity}({original})}} + {\max \mspace{11mu} {{disparity}({original})}}} )}{2}}{{where}\text{:}}{{{Average}\mspace{14mu} {{Disparity}({new})}} = {{{AverageDisparity}({original})} \times {scaling}\mspace{14mu} {factor}}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

where the scaling factor is the ratio of the original display size tothe new display size.

Equation 1 can then be simplified to:

$\begin{matrix}{{{Pixel}\mspace{14mu} {Offset}} = {( \frac{\begin{matrix}{{\max \mspace{14mu} {disparity}({original})} +} \\{\min \mspace{14mu} {{disparity}({original})}}\end{matrix}}{2} ) \times ( {1 - {{scaling}\mspace{14mu} {factor}}} )}} & {{Equation}\mspace{14mu} 1(a)}\end{matrix}$

Thus, a set-top box 104 provided with the minimum and maximumdisparities and the original target screen size can calculate therelevant offset as it is able to interrogate the display 105 for itsdimensions.

Performing this type of global shift provides a general compensation forthe distortion of depth characteristics when the image is scaled to thenew image display size. It will be appreciated that such a linear globalshift is not accurate for all depths, but requires little processingcompared to more complex non-linear shifts. Furthermore, the HVS doesnot always accurately recognise errors in depth. Consequently, such aprocedure may be sufficient to make the depths appear accurate.

This simplistic averaging used as the basis of equation 1 does not takeinto account the non-linear relationship between pixel offset and depthcalculation. Thus, an improved offset can be calculated by using adisparity offset that corresponds to the average depth (and notdisparity) value. This depth based average disparity value can besummarized by equation 2:

$\begin{matrix}{{{{Pixel}\mspace{14mu} {Offset}} = ( {{{Average}\mspace{14mu} {Disparity}\mspace{14mu} {based}\mspace{14mu} {on}\mspace{14mu} {Depth}\mspace{11mu} ({original})} - {{Average}\mspace{14mu} {Disparity}\mspace{14mu} {based}\mspace{14mu} {on}\mspace{14mu} {Depth}\mspace{11mu} ({new})}} )}{{where}:}{{{average}\mspace{14mu} {disparity}\mspace{14mu} {based}\mspace{14mu} {on}\mspace{14mu} {depth}} = {{disparity}( {{average}\mspace{14mu} {depth}} )}}{{where}\text{:}}{{{average}\mspace{14mu} {depth}} = \frac{( {{{depth}( {\min \mspace{14mu} {disparity}} )} + {{depth}( {\max \mspace{14mu} {disparity}} )}} )}{2}}{{where}\text{:}}{{{disparity}\mspace{11mu} ({depth})} = {{i\_ to}{\_ disparity}\mspace{11mu} ( {1 - \frac{1}{({depth})}} )}}{{where}\text{:}}{{{depth}\mspace{11mu} ({disparity})} = \frac{1}{( {1 - {i({disparity})}} )}}{{where}\text{:}}{{i({disparity})} = {{disparity}\mspace{14mu} {in}\mspace{14mu} {pixels} \times \frac{{pixels}\mspace{14mu} {per}\mspace{14mu} {millimeter}}{{eye}\mspace{14mu} {separation}}}}{{where}\text{:}}{{{i\_ to}{\_ disparity}(i)} = \frac{( {i \times {eye}\mspace{14mu} {separation}} )}{{pixels}\mspace{14mu} {per}\mspace{14mu} {millimeter}}}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

where the eye separation is typically fixed at 6.5 cm.

The offset is then calculated such that the average value calculated byequation 2 on the scaled new image display is the same as the valuecalculated by equation 2 for the original target image display.

It will be appreciated that in the above calculations the geometric meanhas been used to illustrate the average calculation. Other calculationsfor the average such as the mode or median, or non-geometric means, canequally be used.

Further improvements can be made by using a knowledge of the number ofpixels that make use of a given disparity value, or depth value. Thisresults in more information being transmitted because the set-top boxuses this information to calculate the offset.

This method for depth perceptive correction of scaled stereoscopicthree-dimensional images of the image stream in accordance with a firstembodiment of the present invention shall now be explained withreference to FIG. 3. In particular, how the above-mentioned process isperformed by the system of FIG. 1 shall be explained in detail.

Firstly, at step 201, maximum and minimum disparity values of thestereoscopic image are obtained, which represent the maximum and minimumdepth of the stereoscopic image.

In this embodiment of the present invention the disparity values aregenerated at the head-end 101, 102. In particular, the head-endgenerates a disparity map, which is a comparison of the offsets ofpixels in the left image of the stereoscopic image with correspondingpixels in the right image of the stereoscopic image. The disparity mapprovides the level of disparity as both positive and negative values,with the positive values representing pixels being perceived asprotruding from the screen, and negative values representing pixelsbeing perceived as being behind the screen. The larger the disparityvalue the larger the depth, i.e. for large disparities, the image willbe perceived as closer to the viewer for positive values, and furtheraway from the viewer for negative values.

The disparity map may be created by various techniques such as automatedcross-correlation procedures, or least squares comparison basedtechniques. Such cross-correlation techniques provide an approximationof the offset, in pixels, by analysing the correspondence between theleft and right images. While the approximation is generally quiteaccurate, errors do occur and it is often desirable to perform somecorrection for those errors. For example, the resultant noisyrepresentation of the disparity can be filtered. A low-pass filter canreduce sudden unusual changes in disparity or depth, which are likely tobe the result of an error. Furthermore, the resolution of the images canbe reduced prior to generating the disparity map in order to reduce theamount of data to be processed and therefore reduce the likelihood oferrors occurring. The disparity map therefore illustrates the disparitybetween the left and right images, which corresponds to the depth of theimage.

In accordance with this embodiment of the invention, the head-end thenanalyses the disparity map in order to determine the maximum disparityand the minimum disparity. This involves searching for the maximumvalue. This can be done by looking at each value in turn and rememberingthe largest. The same is repeated for the smallest value. This is apre-processing step performed prior to transmission by the head-end. Thevalues that are extracted are later used by the set-top box in order tocalculate the shift to apply to the pixels of the stereoscopic imagewhen scaling the image for a new display size. The head-end, therefore,transmits the maximum and minimum disparity values as metadata with thestereoscopic image. Such metadata may be transmitted as a metadatastream synchronized to the video stream carrying the stereoscopic 3Dimage pairs, using techniques such as the one described in EuropeanTelecommunications Standard Institute (ETSI) Technical Specification(TS) 102 823 “Digial Video Broadcasting (DVB); Specification for thecarriage of synchronised auxiliary data in DVB transport streams”. It isnoted that the disparity values and the disparity map correspond to aframe of the image stream. Hence, the metadata is associated with thespecific stereoscopic image frame from which the data is derived.

Once the maximum and minimum disparity values are extracted from thestereoscopic image, and these values are sent across the network, theset-top box obtains the maximum and minimum disparity values from themetadata of the stereoscopic image.

The minimum and maximum disparity values are then averaged as describedabove. This may be an average in the disparity range, or corrected tocorrespond to the average in the depth range (as illustrated by equation2), or may be a median value depending on the accuracy desired, thelevel of information available and the processing power available. Thedisparity result provides a representation of the amount of variation ofthe depth of the stereoscopic image. That is, the maximum and minimumdepth values within the whole image provide an indication of thedisparity range or depth range.

In alternative embodiments, the head-end performs the disparity mapcreation and initial processing of the disparity data prior totransmitting the data in order to reduce the quantity of information tobe transmitted. Furthermore, performing this processing at the head-endalso reduces the processing required at the set-top box. This processingmay involve identification of maximum and minimum values and theaveraging of them, or more complicated processing to produce alternativeaverage values (e.g. rather than the geometric mean used for equations 1and 2, they might use a non-geometric mean, or the mode or the median ofthe values), or reducing the disparity map to a value frequencyhistogram, as indicated by following descriptions.

In yet further alternative embodiments, the set-top box could performall of this processing. That is, the set-top box can generate thedisparity map and extract data from this map as required. Alternatively,the head-end can generate the disparity map and transmit the disparitymap to the set-top box such that the set-top box can use the data of thedisparity map as required for performing the depth adjustments. Decidingwhich of these alternative methods to use depends upon balancing whereit is most preferable to perform the processing, and how important it isto minimise the amount of data transmitted across the network. Thepresent embodiment of the invention being described aims to minimise thedata transmitted across the network, and minimise the amount ofprocessing carried out by the set-top box.

In order to perform the corrections as shown by equation 1 above, ascaling factor, corresponding to a scaling that will scale the imagefrom its original target size to a new size, is also obtained.

Firstly, the set-top box obtains the original target image display sizeat step 203. In this embodiment of the invention, the original targetimage display size is obtained from the metadata of the stereoscopicimage. The depth of the stereoscopic image, that is the offset betweenthe left and right images of the stereoscopic image, is set for beingdisplayed on a particular display size. Such setting of the image sizeensures that the depth is perceived as being accurate by the viewer asalready discussed. Hence, in order to allow for the scaling factor to becalculated, this original target image display size is typicallyprovided in the metadata transmitted by the head-end and thus can beobtained by the set-top box.

While in this embodiment of the invention the original target image sizeis transmitted as metadata, in some alternative embodiments the size oftransmitted image data is standardised such that the original targetimage display size is not provided within the metadata. Such embodimentstherefore reduce the amount of data that is transmitted.

The set-top box obtains the new display size at step 203 by automatedcommunication with the display. For example, an automated informationrequest message is sent from the set-top box to the display. The requestincludes a request for information about the display, which includes thesize of the screen. Once the display returns the screen size informationto the set-top box, this information is stored within a memory of theset-top box. Hence, the set-top box will then store this informationwithin its memory. In alternative embodiments, the set-top box obtainsthe new display size by means of input by the user, e.g. via a userdisplay message in response to which the user is able to input thescreen size.

In the next step of the process, step 204, the set-top box determinesthe ratio of the new image display size to the original target imagedisplay size. This ratio is representative of the scaling required toexpand or shrink the image received by the set-top box to the newdisplay size. Hence, this ratio shall also be referred to as the scalingfactor. It is noted that a factor of less than 1.0 means that the newimage display size is smaller than the original target image displaysize, and larger than 1.0 means the new image display size is largerthan the original target image display size.

At step 205, the set-top box then multiplies the received disparityaverage by one minus the scaling factor to obtain a shifting factor,which is then used at step 206 to laterally shift the left and rightimages of the stereoscopic image with respect to one another. Theshifting factor can be considered as a content specific global shiftvalue, because it provides a global shift, which is a shift of pixels ofthe image, with the amount of shift being based on the content or depthinformation of the stereoscopic image.

In the above-described embodiment of the invention, the left and rightimages are each shifted by half of the shifting factor with respect toone another such that the overall shift equals a shift according to theshifting factor. However, it will be appreciated that either the left orright image could be shifted by the whole shifting factor, or theshifting factor could be split between the left and right images in anyappropriate ratio.

The stereoscopic image is pre-processed ready to be displayed on thedisplay. In particular, the associated depth of the stereoscopic imageis prepared such that, when the image is automatically scaled by beingdisplayed on a display having a different size to the original targetimage display size of the original image data, the depth characteristicsare not unduly distorted.

The process depicted in FIG. 3 is then repeated for further stereoscopicimage frames of an image stream. When processing an image stream thescaling factor typically remains the same, assuming the original targetimage display size and new display size do not change. Hence, thescaling factor can be stored in memory of the set-top box. If required,a simple periodic check can be carried out on the incoming imageinformation to determine if there is a change in the original targetimage display size. In the event that the original target image displaysize does change, the scaling factor is typically recalculated.

Repeated changes of the global shift can be unpleasant to a viewer, andthus the set-top box may change the shift between video frames veryslowly. However, there are certain points where a change in shift ismore acceptable or desirable, such as at scene changes. Scene changescan be detected (e.g. by identifying a major difference in correlationbetween sequential images), and so can be included in the transmittedmetadata to indicate points at which the global shift(s) can be alteredmore freely.

The set-top box 104 is provided with a receiving or input port that isconnected to the receiver 103. The input port feeds received informationto a processor of the set-top box. The processor is able to perform allof the above-mentioned processing, and utilises integrated memorydevices for storing the information utilised during the processing stepsof FIG. 3.

While in the above embodiment of the present invention the set-top boxperforms all of the processing, in alternative embodiments of theinvention all of the processing is performed by the display. In yetfurther embodiments of the invention the display and set-top box may beintegrated components and thus the processing is performed by sharedprocessing components of the display and set-top box.

In yet further embodiments of the invention, more complex operations canbe performed in order to better compensate for the errors in depthcharacteristics that occur when scaling stereoscopic images. Thesefurther embodiments are set out below.

For each of the further embodiments greater amounts of metadata can beused beyond the simple minimum and maximum disparity, up to andpotentially including a full per pixel disparity map. The disparity mapsmay be sent as metadata accompanying the stereoscopic image information,or alternatively they may be generated by the set-top box.

In a first alternative embodiment of the present invention, thenon-linearity of perceived depth when varying image size is compensatedfor by using methods for minimising the error in depth scaling over therange of depths. Many methods can be used for this. For example,iterative methods can be used to attempt to minimise the errors seenacross the range.

For any given image or scene, each potential disparity or depth valuemay occur a given number of times (which may be zero), and thisrepresents the usage count of that value. Thus a histogram (or similar)can be generated which represents the distribution of the usage of thedifferent disparity (or depth) values. To be of use for the followingstages, these values can typically be measured not in pixels but infractions of the human eye separation. Thus, when the display size isscaled, and an offset is applied, this disparity (or depth) value usagehistogram is changed. This is caused by two properties of the displaysize scaling; firstly the overall represented range from maximum tominimum is scaled resulting in a smaller or larger overall range (as weare no longer measuring in pixels but in fractions of a fixed size), andsecondly the scaling of the image results in different usage counts forthe values.

An iterative method can be used to compare the original and scaledoffset histograms. Since only the offset may be varied, a differenthistogram can be generated for each offset value. This new histogram canbe compared against the original histogram, and the best match can beused to select the desired offset. The comparison can use any of a rangeof mechanisms, such as the sum of absolute differences (SAD) or theleast squares mechanisms.

As the importance of certain depths will vary, use of selective rangesof values can further improve the method. For instance a relatively flatscene background that is static (i.e. the maximum depth is not changing)may well not matter as much as the central character(s) of the scene.Thus further versions that exclude or minimise the importance of certaindepth values (mostly likely to be a static background) would furtheralter the identified shift correction factor by excluding this depthvalue, or set of depth values from the overall correction. In manycases, this can be handled by excluding the furthest back values (i.e.the greatest positive disparity values) and optimising for the remainingvalues using the comparison outlined above, but operating on the reducedhistogram data.

The embodiments described above provide a global shift to pixels of thestereoscopic image. In a further embodiment of the invention, thestereoscopic image is divided up into regions. A shift factor isdetermined for one or more regions, and each of the one or more regionsis then shifted in accordance with its specific shifting factor. Thisembodiment of the invention shall now be described in more detail.

Firstly, the disparity map is obtained, either from metadata associatedwith the stereoscopic image or by generating the map by performing adisparity cross-correlation between the left and right images of thestereoscopic image. Then the stereoscopic image is split into regions byperforming any of numerous methods such as object identification orregion splitting or region growing. One simple method is to process thedisparity map to reduce the number of values, for example by dividingall values by a given constant, such that, for example, the disparitymap is reduced to binary values (0 or 1). The resultant disparity map isscanned from the top line downwards until a point is reached where themajority of the values change from a 1 to a 0 (or vice versa. From this,two regions, a “top half” and a “bottom half” are identified, anddifferent offsets can be calculated for each half. Clearly, state of theart region identification, splitting or growing algorithms will producesignificantly better regions that in turn will have better visualresults.

Other non-content based region splitting techniques may also beutilized. In this case, the image is divided up into regions, such as16×16 pixel blocks, and each block is its own region. Such techniquesmay break the image up into regions of similar disparity. It is notedthat regions of similar disparity often correspond to a particularobject of the image. Hence, this embodiment of the invention thereforeeffectively provides different shifts to different objects within theimage.

For some forms of content, the generation process can easily provideregion information for assets in the scene, or closely grouped assets inthe scene. One mechanism for achieving this is to generate, for pixelsin the image pair, the asset that is rendered in that pixel and placingthis into an asset map of the same resolution as the image pair. Thisasset map can be processed further by grouping closely related assets(e.g. all the leaves on a tree). The asset map will then have acollection of regions identified by a common asset ID, and this can beused as a region map, or as an initial starting region map.

Once the image is split into regions the various techniques fordetermining a global shifting factor can be applied to one or more ofthe regions. Since the regions have similar disparities, errors in thenew depth values are minimised.

Any of the techniques identified above can be used to identify theoffset to be used for the region identified. When operating on a regionand using the head-end to identify regions, the geometry of the region,as identified by the head-end processes just described, would betransmitted together with the required disparity information for theoffset calculation mechanism.

Human depth perception is very accurate for objects that seem to beclose to the viewer. However, as depth of objects increases, theaccuracy of perception is reduced. Hence, in further embodiments of theinvention, techniques for adjusting the depth can be applied whichemploy a degree of algorithmic complexity sufficient for the requiredaccuracy of perception at that depth. A coding scheme for conveying thedepth adjustment can be similarly designed.

In further embodiments of the present invention, the boundaries betweenregions can be smoothed. Performing smoothing reduces sudden changes indepth characteristics, which if not corrected for may look unnaturalwhen displayed. Such smoothing can be achieved by applying curve-fittingtechniques to disparity data that bridges regions.

There are numerous techniques from computer graphics that take a rangeof sample values and provide a range of methods for interpolating valuesand achieving behaviour equivalent to smoothing. Treating the offsetvalue(s) as input values to a curve-fitting algorithm, such as (forexample) the non-uniform rational B-spline(http://en.wikipedia.org/wiki/NURBS) can be used. Where such algorithmsallow for further optional values to control the fitting, these valuescan take standard default values or could be controlled by furthertransmitted data. Those skilled in the art will appreciate that manyalternative algorithms could be used to achieve the same effect.

While each of these more complex embodiments of the present inventionmay increase the processing requirements of the set-top box, it will beappreciated that such functionality could be implemented effectively bya standard Graphics Processing Unit (GPU) or alternatively by a siliconchip specifically designed for performing such functionality.

The embodiments described above utilise additional information togenerate a shifting factor (or shifting factors) to be applied to thecontent. The value of the correction factor(s) are determined inconjunction with the scaling factor and hence in conjunction with thesize of the display device. As such, each differently sized displaydevice will generate a different shifting factor(s). In furtherembodiments, it may be desirable to limit the maximum shifting factor(positive or negative) to keep the content within a physiologicallycomfortable range, or retain a desired artistic effect. To this end,additional information can be transmitted that represents a maximumallowable shifting factor (positive or negative) that can be applied.

Each of the above mentioned embodiments of the invention have beendescribed in accordance with the system as depicted in FIG. 1. Thesystem of FIG. 1 relates to a standard television network, in which thethree-dimensional image is transmitted through the air between atransmitter and receiver. However, it will be appreciated that thepresent invention could equally apply to a system in whichsatellite-based transmission is utilised, or the transmission is madeover the Internet. Furthermore, while the set-top box and television aredepicted as separate items, these could be integrated within a singleunit, and if the images are sent over the Internet, the set-top box 104and display 105 could be replaced by a computer.

In yet further embodiments of the invention, the stereoscopic image datais not received over a network but is instead received from a datastorage device, such as a DVD. In such an embodiment, the storage devicewill typically store the stereoscopic image data in the form of an imagestream. The storage device may also provide metadata relating to theoriginal target image display size, as well depth information such asmaximum and minimum depth values for one or more frames, or a depth mapfor one or more frames. In this further embodiment of the presentinvention, the functionality of the set-top box may be performed withinthe storage device reader, such as a DVD player, or within the displaydevice itself. Furthermore, all of these functionalities may beincorporated in a computer.

While the above embodiments have been described with respect to use withtelevisions or computer monitors as the display device it will beappreciated that the present invention could be used with any displaymeans, such as a mobile phone screen, or a cinema/projector screen.

In the above description, mechanisms for correcting depth perspectivecorrection for two images were described. The same mechanisms can extendto multiple images, where different correction factors are applied toeach image, or where they are interpolated between the images based on aknowledge of the relationship between the images.

The present invention can be implemented in dedicated hardware, using aprogrammable digital controller suitably programmed, or using acombination of hardware and software.

Alternatively, the present invention can be implemented by software orprogrammable computing apparatus. This includes any computer, or suchlike. The code for each process in the methods according to theinvention may be modular, or may be arranged in an alternative way toperform the same function.

Each of the functionalities of the invention can in whole, or in part,be implemented by the combination of a processor and associated memory,or by a standard computer system. Furthermore, functions describedherein as being implemented as part of a single unit may be providedseparately, communicatively coupled across a network.

The present invention can encompass a carrier medium carryingmachine-readable instructions or computer code for controlling aprogrammable controller, computer or number of computers as theapparatus of the invention. The carrier medium can comprise any storagemedium such as a floppy disk, CD ROM, DVD ROM, hard disk, magnetic tape,or programmable memory device, or a transient medium such as anelectrical, optical, microwave, RF, electromagnetic, magnetic oracoustical signal. An example of such a signal is an encoded signalcarrying a computer code over a communications network, e.g. a TCP/IPsignal carrying computer code over an IP network such as the Internet,or an intranet, or a local area network.

Throughout this document, reference to the depth of thethree-dimensional image refers to the distance from the viewer to thepoint at which they perceive the object. Thus, an object that isperceived at the same location on the screen would have a depth that isequivalent to the distance to the screen. An object that is perceivedbehind the screen would have a greater depth, and an object that isperceived in front of the screen would have a lesser depth than thedistance to the screen.

Reference to the maximum depth throughout this document refers to theperceived depth associated with the object that is perceived furthestfrom the viewer. This would normally, but not necessarily, be at orbehind the screen.

Reference to the minimum depth throughout this document refers to theperceived depth of the object that is perceived closest to the viewer.This need not be in front of the screen, but it may be perceived as infront of the screen.

In a stereoscopic image pair discussed throughout this document, anobject is represented at potentially different locations in the left andright images that make up the stereoscopic pair. The term disparityrefers to a measure of the offset between these locations. Pixels willtypically have a disparity value. A negative disparity means that thelocation of the object in the right image is to the left of the locationof the object in the left image. This means that the object is perceivedin front of the screen, or that the depth of the object is less than thedepth of the screen, or that the object is closer to the viewer than thescreen. A positive disparity means the opposite, and that the object isperceived behind the screen.

The disparity can be measured in pixels or, for a given display size,converted into a distance measurement in metres using the pixels permillimetre value for the display.

Reference to the maximum disparity is referring to the disparity withthe largest positive value, and corresponds to the object that is thefurthest from the viewer.

Reference to the minimum disparity refers to the mathematical minimumvalue of the disparity values, i.e. it is the value with the mostnegative value, or if no negative values are present, it is the smallestpositive value (treating zero as the smallest possible positive value).

The disparity domain is the domain in which the offset of the images ofthe stereoscopic image are considered in pixels per millimetre. Incontrast, the depth domain is the domain in which the perceived depth ofthe images of the stereoscopic image is considered in terms of distancefrom the viewer.

When reference is made throughout this document to an object beingbehind the screen, this means that due to the characteristics of thestereoscopic pair, the object is perceived as being on the opposite sideof the screen to that of the viewer. Conversely, reference to an objectbeing in front of the screen refers to an object that is perceived asbeing closer to the viewer than the screen.

It is appreciated that various features of the invention that are, forclarity, described in the contexts of separate embodiments may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention that are, for brevity, described in thecontext of a single embodiment may also be provided separately or in anysuitable subcombination.

It will be appreciated by persons skilled in the art that the presentinvention is not limited by what has been particularly shown anddescribed hereinabove. Rather the scope of the invention is defined onlyby the claims.

1. A method for adjusting depth characteristics of a three-dimensionalimage for correcting for errors in perceived depth when scaling thethree-dimensional image, the method comprising: receivingthree-dimensional image information comprising a stereoscopic imageincluding a first image and a second image, the stereoscopic imagehaving depth characteristics associated with an offset of the first andsecond images; determining a scaling factor indicative of a scaling forconverting the stereoscopic image from an original target size to a newsize; determining a shifting factor for varying the depthcharacteristics, the shifting factor indicative of a relative shift tobe applied between the first and the second images, wherein the shiftingfactor is determined in accordance with the scaling factor and at leastone depth parameter derived from the depth characteristics; andperforming the relative shift between the first and second images inaccordance with the shifting factor for adjusting the offset of thefirst and second images.
 2. The method according to claim 1, wherein theat least one depth parameter used to determine the shifting factorrelates to a variation in the depth of the stereoscopic image.
 3. Themethod according to claim 2, wherein the variation corresponds to anaverage variation in the depth of the stereoscopic image.
 4. The methodaccording to claim 3, wherein the average variation is determined inaccordance with the following equation:(maximum disparity+minimum disparity/2)×(1−scaling factor) wherein thedisparity is a measure of an offset between a pixel in the first andsecond images of the stereoscopic image.
 5. The method according toclaim 3, wherein the average variation is determined in the depth domainand transformed to the disparity domain for determining the shiftingfactor.
 6. The method according to claim 1, wherein the shifting factoris determined by an iterative process for minimising errors in theoffset of the first and second images of the stereoscopic image whenscaling the stereoscopic image.
 7. The method according to claim 1,wherein the iterative process utilises information related to the usageof different depth values.
 8. The method according to claim 1, whereinthe at least one depth parameter used to determine the shifting factorincludes a parameter indicating a most common depth of the depthcharacteristics of the stereoscopic image.
 9. The method according toclaim 1, wherein the three-dimensional image information includes the atleast one depth parameter.
 10. The method according to claim 1, whereinthe stereoscopic image comprises a plurality of regions and the step ofdetermining the shifting factor further comprises determining a shiftingfactor for one or more of the regions, and the step of performing arelative shift further comprises performing a relative shift for each ofthe one or more regions in accordance with the shifting factordetermined for each respective one or more region.
 11. The methodaccording to claim 10, wherein the step of determining the shiftingfactor further comprises splitting the stereoscopic image into regions.12. The method according to claim 10, wherein the three-dimensionalimage information includes information defining the regions of thestereoscopic image.
 13. The method according to claim 10, furthercomprising performing a smoothing operation on boundaries betweenregions.
 14. The method according to claim 13, wherein the smoothingoperation involves low-pass filtering.
 15. The method according to claim1 further comprising: receiving further three-dimensional imageinformation comprising a further stereoscopic image including a firstimage and a second image, the further stereoscopic image having depthcharacteristics associated with an offset of the first and second imagesof the further stereoscopic image, wherein the stereoscopic image andthe further stereoscopic image correspond to frames of a stream of imageinformation and the further stereoscopic image corresponds to a frame ofthe stream preceding the further stereoscopic image; and performing asmoothing operation on the depth characteristics in order to smooth thetransition of depth characteristics between frames.
 16. The methodaccording to claim 15, wherein the smoothing operation comprisesapplying a low pass filter to the depth information.
 17. The methodaccording to claim 1, wherein the depth parameters are determined inaccordance with a depth map included in the three-dimensional imageinformation, the depth map indicating a relative offset between thefirst and the second images at a plurality of points across thestereoscopic image.
 18. The method according to claim 1, furthercomprising generating a depth map from the stereoscopic image, whereinthe depth parameters are determined in accordance with the depth map,the depth map indicating a relative offset between the first and thesecond images at a plurality of points across the stereoscopic image.19. The method according to claim 1, wherein the three-dimensional imageinformation is received over a network from a head-end.
 20. The methodaccording to claim 1, wherein the first and second images are eachshifted relative to one another by half of the shifting factor forcorrecting depth perception due to scaling of the three-dimensionalimage.
 21. An apparatus for adjusting depth characteristics of athree-dimensional image for correcting for errors in perceived depthwhen scaling the three-dimensional image, the apparatus comprising: aninput port arranged to receive three-dimensional image informationcomprising a stereoscopic image including a first image and a secondimage, the stereoscopic image having depth characteristics associatedwith an offset of the first and second images; and a processor arrangedto: determine a scaling factor indicative of a scaling for convertingthe stereoscopic image from an original target size to a new size;determine a shifting factor for varying the depth characteristics, theshifting factor indicative of a relative shift to be applied between thefirst and the second images, wherein the shifting factor is determinedin accordance with the scaling factor and at least one depth parameterderived from the depth characteristics; and perform the relative shiftbetween the first and second images in accordance with the shiftingfactor for adjusting the offset of the first and second images.
 22. Theapparatus according to claim 21, wherein the at least one depthparameter used to determine the shifting factor relates to a variationin the depth of the stereoscopic image.
 23. The apparatus according toclaim 22, wherein the variation corresponds to an average variation inthe depth of the stereoscopic image.
 24. The apparatus according toclaim 23, wherein the processor determines the average variation inaccordance with the following equation:$( \frac{{{maximum}\mspace{14mu} {disparity}} + {{minimum}\mspace{14mu} {disparity}}}{2} ) \times ( {1 - {{scaling}\mspace{14mu} {factor}}} )$wherein the disparity is a measure of an offset between a pixel in thefirst and second images of the stereoscopic image.
 25. The apparatusaccording to claim 23, wherein the processor determines the averagevariation in the depth domain and transforms the average variation tothe disparity domain for determining the shifting factor.
 26. Theapparatus according to claim 21, wherein the processor determines theshifting factor by an iterative process for minimising errors in theoffset of the first and second images of the stereoscopic image whenscaling the stereoscopic image.
 27. The apparatus according to claim 21,wherein the iterative process utilises information related to the usageof different depth values.
 28. The apparatus according to claim 21,wherein the at least one depth parameter used to determine the shiftingfactor includes a parameter indicating a most common depth of the depthcharacteristics of the stereoscopic image.
 29. The apparatus accordingto claim 21, wherein the three-dimensional image information includesthe at least one depth parameter.
 30. The apparatus according to claim21, wherein the stereoscopic image comprises a plurality of regions andthe processor determines a shifting factor for one or more of theregions, and performs the relative shift for each of the one or moreregions in accordance with the shifting factor determined for eachrespective one or more region.
 31. The apparatus according to claim 20,wherein the processor determines the shifting factor by splitting thestereoscopic image into regions.
 32. The apparatus according to claim20, wherein the three-dimensional image information includes informationdefining the regions of the stereoscopic image.
 33. The apparatusaccording to claim 20, wherein the processor is further arranged toperform a smoothing operation on boundaries between regions.
 34. Theapparatus according to claim 23, wherein the smoothing operationinvolves low-pass filtering.
 35. The apparatus according to claim 2,wherein the processor is further arranged to: receive furtherthree-dimensional image information comprising a further stereoscopicimage including a first image and a second image, the furtherstereoscopic image having depth characteristics associated with anoffset of the first and second images of the further stereoscopic image,wherein the stereoscopic image and the further stereoscopic imagecorrespond to frames of a stream of image information and the furtherstereoscopic image corresponds to a frame of the stream preceding thefurther stereoscopic image; and perform a smoothing operation on thedepth characteristics in order to smooth the transition of depthcharacteristics between frames.
 36. The apparatus according to claim 25,wherein the smoothing operation comprises applying a low pass filter tothe depth information.
 37. The apparatus according to claim 21, whereinthe processor determines the depth parameters in accordance with a depthmap included in the three-dimensional image information, the depth mapindicating a relative offset between the first and the second images ata plurality of points across the stereoscopic image.
 38. The apparatusaccording to claim 21, wherein the processor is further arranged togenerate a depth map from the stereoscopic image, wherein the depthparameters are determined in accordance with the depth map, the depthmap indicating a relative offset between the first and the second imagesat a plurality of points across the stereoscopic image.
 39. Theapparatus according to claim 21, wherein the three-dimensional imageinformation is received over a network from a head-end.
 40. Theapparatus according to claim 21, wherein the processor is arranged toshift the first and second images relative to one another by half of theshifting factor for correcting depth perception due to scaling of thethree-dimensional image. 41-42. (canceled)
 43. An apparatus foradjusting depth characteristics of a three-dimensional image forcorrecting for errors in perceived depth when scaling thethree-dimensional image, the apparatus comprising: means for receivingthree-dimensional image information comprising a stereoscopic imageincluding a first image and a second image, the stereoscopic imagehaving depth characteristics associated with an offset of the first andsecond images; and means for determining a scaling factor indicative ofa scaling for converting the stereoscopic image from an original targetsize to a new size; means for determining at least one a shifting factorfor varying the depth characteristics, the at least one shifting factorindicative of a relative shift to be applied between the first and thesecond images, wherein the at least one shifting factor is determined inaccordance with the scaling factor and at least one depth parameterderived from the depth characteristics; and means for performing therelative shift between the first and second images in accordance withthe shifting factor for adjusting the offset of the first and secondimages.